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The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. ' 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S. C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 09 December 2002 . 
2a)Q This action is FINAL. 2b)IEI This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) E3 Claim(s) 1,2 and 13-61 is/are pending in the application. 

4a) Of the above claim(s) 1,2,27,49 and 50 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |El Claim(s) 13-26.28-48 and 51-61 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) IEI The drawing(s) filed on 23 October 2000 is/are: a)E3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

11) D The proposed drawing correction filed on is: a)Q approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

aO All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) ^ Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1 ) S Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 

2) SI Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-152) 

3) ^ Information Disclosure Statement(s) (PTO-1449) Paper No(s) 6. 8. 9 . 6) □ Other: 

U.S. Patent and Trademark Office "™ 

PTO-326 (Rev. 04-01 ) Office Action Summary Part of Paper No. 1 5 
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Election/Restrictions 



1 . Applicant's election without traverse of Group III (claims 13-26, 28-48, 51-61 ) in 
Paper No. 14 is acknowledged. 

Claims 1-2, 27 and 49-50 are withdrawn from further consideration pursuant to 
37 CFR 1 .142(b) as being drawn to a nonelected invention, there being no allowable 
generic or linking claim. Election was made without traverse in Paper No. 14. 



2. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

3. Claims 17, 51-56 is rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

Claim 17, at line 2, "the provided images" lacks clear antecedent basis. It is not 
clear what "provided images" are being referred to. 

The scope of claims 51-56 is unclear. Claim 51 is drawn to an "application." An 
application is taken to mean a software program, as is known in the art. However, the 
claims recite various structural elements, such as "means," "systems," "storage," etc., 
which are more appropriate for an apparatus or system. 



Claim Rejections - 35 USC §112 



Application/Control Number: 09/344,814 
Art Unit: 2623 



Page 3 



Claim Rejections - 35 (JSC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

6. Claims 13-26, 28-48 and 51-61 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of U.S. Patent 6,331 ,858 to Fisher and U.S. Patent 
6,333,749 to Reinhardt et al. (hereinafter "Reinhardt"). 

As to claim 13, Fisher discloses a method for generating remote presentations of 
products in situ for a user comprising the steps of: 

a) prompting the user to provide a representation of a space in which a product is 
to be viewed in context (column 2, lines 47-50; column 5, lines 45-46); 

b) accepting a product representation representing a product for which a three- 
dimensional geometric model exists or can be created (column 5, line 43); 

e) accepting an input of a product location, the product location being a location 
in the space where the product is to be displayed in situ (column 5, lines 48-49); 
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g) forming an image of the space where the modified image has been modified to 
show the identified product in situ in the image with a proper proportion and perspective 
(column 1, lines 40-41; column 2, lines 11-13; column 5, lines 48-49). 

Fisher does not disclose acquiring a digital image of the space, or the steps of c), 
d), f) and g) (as claimed). However, in an analogous environment, Reinhardt discloses: 

acquiring a digital image of a space (column 4, lines 1-2; column 5, lines 5-8); 

c) determining at least one dimension reference, wherein a dimension reference 
is a measurement of a distance in the space corresponding to a separation of two points 
on the digital image (column 11, lines 19-25); 

d) calculating, from the image, the at least one dimension reference and 
reference information, camera parameters for a camera in the space from which the 
image was captured, wherein the camera parameters include at least a camera position 
and a focal length (abstract, lines 5-6); 

f) transforming the geometric model of a product based on at least the camera 
position and the product location to form a transformed geometric model (column 3, line 
67 to column 4, line 4; column 7, lines 62-65); and 

g) combining the transformed geometric model and the digital image to form a 
modified image of the space (column 5, lines 25-27). 

Reinhardt's technique would provide more realistic images of the space 
containing the products in situ, therefore it would have been obvious to one of ordinary 
skill in the art to modify Fisher's invention according to Reinhardt. 
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With regard to claim 14, Fisher discloses the method of claim 13, wherein the 
step of accepting a product representation comprises the steps of: 

prompting the user to identify a product of interest from a set of products (column 
2, lines 60-61 ; column 5, lines 46-50; allowing the user to select implies prompting of 
the user); 

searching a collection of product representations to locate a member of the 
collection that matches the product identified by the user (column 2, lines 60-61 ; 
selection of one of a plurality of products implies that the a collection of such products is 
searched and located, and matches that identified by the user); and 

accepting the matching product representation (column 2, lines 63-64; column 5, 
lines 48-51). 

Regarding claim 15, repeating steps b), e) and f) for second and subsequent 
selected products would have been an obvious extension to the combined Fisher- 
Reinhardt method. Note that to only perform the steps for one product would defeat the 
purpose of the invention, and would make the invention essentially useless for retail 
transactions over the Internet. 

As to claim16, Fisher teaches prompting the user to acquire a three-dimensional 
product of interest to the user (e.g., column 5, lines 65-66; column 6, line 7). In 
combining Fisher and Reinhardt as discussed above with regard to claim 1 , a three- 
dimensional geometric model would be acquired (e.g., see Reinhardt, column 4, line 1). 

Regarding claim 17, repeating step a) for second and subsequent images of the 
space and using each of the provided images would have been an obvious extension to 
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the combined Fisher-Reinhardt method. Keeping in mind that Fisher allows different 
viewpoints of the space (column 2, lines 11-13), to only perform the step for one product 
would defeat the purpose of the invention, and would make the invention essentially 
useless for retail transactions over the Internet. 

As to claim 18, Fisher, as modified by Reinhardt discloses the method of claim 
13, wherein the modified image of the space is a two-dimensional view of a three- 
dimensional geometric model (note Fisher's column 1 , lines 44-45; and the 3D 
geometric model taught by Reinhardt), the method further comprising a step of moving 
a camera position of the modified image of the space to simulate moving around in the 
space (e.g., Fisher, column 2, lines 11-13). 

With regard to claim 19, Fisher discloses the method of claim 13, wherein the 
step of prompting the user to identify the product from a set of products is performed 
using a commerce server that serves product models and further comprising a step of 
transmitting the digital image to the commerce server (column 3, line 27). 

As to claim 20, Fisher discloses the method of claim 13, wherein the step of 
prompting the user to identify the product from a set of products is performed using a 
commerce application that receives product models and further comprising a step of 
providing the digital image to the commerce application (column 5, line 66 to column 6, 
line 2). 

With regard to claim 21 , Reinhardt further teaches that the camera parameters 
include camera position, camera rotation (orientation), focal length (abstract, lines 5-6). 
Reinhardt does not explicitly mention that a camera parameter includes center of 




• 
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projection. However, the Examiner takes Official Notice that the camera parameter 
center of projection is well known. It would have been obvious to calculate this camera 
parameter because it would provide an enhanced 3D scene. 

As to claim 22, Reinhardt further teaches that the reference information includes 
correspondences between two-dimensional image features and three-dimensional 
structures (column 11, lines 19-25; the points are in the 2D image, distance is taken in 



As to claim 23, Reinhardt further teaches that the step of determining the at least 
one dimension reference is a step of inputting the at least one dimension reference, 
wherein a dimension reference is a measurement of a distance in the space 
corresponding to a separation of two points on the digital image (column 11, lines 19- 



As to claim 24, Reinhard further teaches that the step of determining the at least 
one dimension reference is a step of assuming a default scale and using the default 
scale to determine the at least one dimension reference column 1 1 , lines 19-25, the 
default scale is implied by the ruler distance). 

With regard claim 25, Fisher discloses the method of claim 13, wherein the three- 
dimensional model for a product is a planar representation of an object and a texture 
map to be applied to a surface of the object (column 4, lines 39-40). 

With regard to claim 26, reference is made to the discussion provided above for 
claims 13 and 19. Specific structure, such as storage, engines, etc., are considered 
inherent in Fisher and Reinhardt, given that they are computer based systems, and any 



3D). 



25). 
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computer programmed to perform the indicated method would provide the required 
structure. 

Claim 28 is similar to claim 13. The discussion provided above for claim 13 is 
applicable to claim 28. 

With regard to claim 29, Fisher discloses the method of claim 28, wherein the 
product representation includes a texture map associated with the product (column 4, 
lines 13-14,39-40). 

As to claim 30, Fisher is silent with regard to the product representation includes 
at least a product digital image, captured by a product image capture device positioned 
to capture a digital image of the product. However, the Examiner contends that the 
images of the products must be captured using some kind of device. Further, Reinhardt 
teaches utilizing a digital camera to capture a digital image of objects (column 7, lines 
18-19). It would have been obvious to utilize a digital camera as taught by Reinhardt in 
Fisher's invention because of the convenience provided by digital cameras. 

As to claim 31 , Reinhardt further teaches that the product representation includes 
a capture position, capture angle of rotation and focal length (abstract, lines 5-7; note 
that these parameters are derived from the images, implying that the representation 
includes them). 

With regard to claim 32, Reinhardt further teaches that the product 
representation includes product image capture parameters including at least a 
representation of the position of the product image capture device when the digital 
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image of the product was captured (abstract, lines 5-7; note that these parameters are 
derived from the images, implying that the representation includes them). 

With regard to claim 33, Reinhardt further teaches that the product image capture 
parameters include projections of geometric elements from the product onto the product 
digital image (column 5, lines 25-28). 

With regard to claim 34, Reinhardt further teaches that the capture parameters 
for the image of the scene include projections of geometric elements from the scene 
onto the scene digital (column 5, lines 25-39). 

As to claim 35, Reinhardt further teaches that the capture parameters for the 
image of the scene include an angle of rotation for the image capture device in the 
space (abstract, lines 5-7). 

As to claim 36, Fisher, as modified by Reinhardt, discloses the method of claim 
28, wherein the step of inputting a product representation is preceded by the steps of: 
capturing a product digital image using the product image capture device (note as 
above, Reinhardt provides for the product image capture device); and 
calculating, from the product digital image, product image capture parameters that 
include at least a representation of the product image capture position and a 
representation of the product image capture focal length (note as above, Reinhardt 
teaches this, e.g., in the abstract, lines 5-7). 

As to claim 37, Reinhardt further teaches that the step of transforming results in 
the one or more transformed digital images sharing common capture parameters, the 
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common capture parameters including at least a capture position (column 11, lines 1-7; 
column 12, lines 12-14). 

As to claim 38, Fisher discloses the method of claim 28, further comprising a step 
of prompting the user to identify the product from a set of products for which a three- 
dimensional geometric model exists or can be created (column 2, lines 60-61; column 5, 
lines 46-50; allowing the user to select implies prompting of the user). 

As to claim 39, see the discussion above for claim 23. 

As to claim 40, repeating step of combining would have been an obvious 
extension to the combined Fisher-Reinhardt method. To only perform the step for one 
product would defeat the purpose of the invention, and would make the invention 
essentially useless for retail transactions over the Internet. 

With regard to claim 41 , see the remarks above for step d) of claim 13. 

With regard to claim 42, Fisher discloses a method for generating a combined 
image that simulates a product being positioned at a location where a consumer is 
considering placing the product, without physically placing the product at the location, 
the method comprising the steps of- 

a) prompting the consumer to a representation of the location (column 2, lines 
47-50; column 5, lines 45-46); 

b) generating a location representation of the location (column 2, lines 47-50; 
column 5, lines 45-46) items at the location visible in the digital image of the location 
(column 5, lines 48-50); 
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c) prompting the consumer to select a product from among a plurality of products 
(column 2, lines 60-61; column 5, lines 46-50; allowing the user to select implies 
prompting of the user); 

d) retrieving a product representation for a selected product (column 5, lines 46- 

50). 

Fisher does not disclose acquiring a digital image of the location, generating 
geometric elements representing items at the location visible in the digital image of the 
location, and that the product representation includes at least a digital image of the 
product and a representation of the position of a product image capture device when the 
digital image of the product was captured. 

However, in an analogous environment, Reinhardt discloses: 

acquiring a digital image of a location (column 4, lines 1-2; column 5, lines 5-8); 

generating geometric elements representing items at a location visible in the 
digital image of the location (column 5, lines 25-28); 

product (object) representation includes at least a digital image of the product 
and a representation of the position of a product image capture device when the digital 
image of the product was captured (abstract, lines 5-7). 

Reinhardt 1 s technique would provide more realistic images of the space 
containing the products, therefore it would have been obvious to one of ordinary skill in 
the art to modify Fisher's invention according to Reinhardt. 

As to claim 43, Reinhardt further teaches that the location representation 
includes location capture parameters of a location capture device, wherein the location 



Application/Control Number: 09/344,814 Page 12 

Art Unit: 2623 

capture parameters include at least a representation of a position of the location capture 
device (abstract, lines 5-7). 

A to claim 44, Fisher discloses the method of claim 42, further comprising the 
step of: e) prompting the consumer to indicate where, at the location, the product is to 
be considered (column 5, lines 44-50). Fisher, as modified by Reinhardt teaches f) 
combining the location representation and the product representation after transforming 
at least one of the representations to form the combined image showing the product in 
situ with a proper proportion and perspective (e.g., Reinhardt, column 5, lines 25-27). 

As to claim 45, Fisher discloses that the product representations are 3D (column 
5, lines 65-66) and a texture map (column 4, lines 13-14). Fisher does not disclose that 
the product representation includes at least a three-dimensional geometric model. 
Reinhardt teaches using three-dimensional geometric models for objects (e.g., see 
abstract, column 5, lines 25-28). Given that Fisher's objects are 3D, it would have been 
obvious to utilize three-dimensional geometric models as taught by Reinhardt because 
this would simplify and speed up rendering. 

As to claim 46, Fisher combined with Reinhardt discloses the method of claim 42, 
wherein the method is performed by a commerce server and a consumer computer 
(column 3, lines 27-30), the method further comprising the steps of- 

e) sending a commerce application from the commerce server to the consumer 
computer (Fisher: column 5, line 63 to column 6, lines 5); 

f) executing the commerce application on the consumer computer, wherein the 
commerce application performs steps a) and c) (Fisher: column 6, lines 2-3); 
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g) using the commerce application to model geometric elements visible in the 
digital image of the location (this would occur in the combined Fisher-Reinhardt 
method); and 

h) generating capture parameters for the digital image of the location from the 
geometric elements using the commerce application, the capture parameters including 
a capture location and a focal length (Reinhardt: abstract, lines 5-7). 

As to claim 47, Fisher discloses the method of claim 42, wherein the method is 
performed by a commerce server and a consumer computer and wherein the step of 
retrieving the product representations is a step of retrieving the product representations 
from the commerce server (column 3, lines 28-30; the server holds the product 
representations). 

As to claim 48, Fisher discloses the method of claim 42, wherein the method is 
performed by a commerce server and a consumer computer and that the step of 
retrieving the product representations is a step of retrieving the product representations 
from a product representation server (column 28-30, the web server is a product 
representation server). 

With regard to claim 51 , reference is made to the remarks provide above for 
claims 13, 19 and 26. 

As to claim 52, remarks analogous to those provided above for claim 14. 

Regarding claims 53-56, having the various components in either the server or 
the client is seen as a decision based upon designer preference. A designer would 
have a particular component in either the client or the server based on his or her needs 
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for the given application. This is not considered a patentable difference from the prior 
art. 

As to claim 57, Fisher discloses a method of generating an image of a simulated 
space, wherein the space is simulated in that it shows an object placed in a scene, 
where the object is a physical object and the scene is a physical location, the method 
comprising the steps of: 

acquiring one or more representations of the scene , wherein the scene does not 
contain, at the time of acquisition, the object at a desired object location in the scene 
(column 2, lines 47-50); 

acquiring an object representation of the object that is to be simulated in the 
scene, where the object representation is at least an image of the object (column 5, 
lines 43, 48-49); and 

combining the scene representation and the object representation to form the 
image of the simulated space (column 5, lines 48-49). 

Fisher does not disclose that the representation of the scene is an image, or the 
generating step. However, Reinhardt teaches acquiring an image of a scene (column 4, 
lines 1-2; column 5, lines 5-8), and generating a scene representation that includes a 
model of an image acquisition device that captured at least one of the one or more 
scene images acquired in the step of acquiring one or more images of the scene, the 
model including at least a representation of a position of the image acquisition device in 
a space containing the at least scene image when the at least one scene image was 
captured (abstract, lines 5-7). 
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Reinhardt's technique would provide more realistic images of the space 
containing the products in situ, therefore it would have been obvious to one of ordinary 
skill in the art to modify Fisher's invention according to Reinhardt. 

As to claim 58, Reinhardt further teaches that the image is taken from an image 
capture device (column 7, lines 12-20). 

As to claim 59, Fisher further discloses the method of claim 57, wherein the 
object representation includes a texture map (column 4, lines 13-14). 

As to claim 60, Fisher discloses the method of claim 57,further comprising a step 
of obscuring a portion of the scene with the object where the object representation is 
constrained such that the positioning or size of the object image is dependent on the 
positioning or size of an object present in the scene (e.g., see Fig.3, element 302; Fig. 3, 
element 302). 

With regard to claim 61 , reference is made to the discussion provided above for 
claims 44 and 45. 

References Cited 

7. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

U.S. Patent 5,053,956 to Donald et al. discloses an interactive system for retail 
transactions which can show a selected product against a background image. 
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U.S. Patent 5,970,471 to Hill discloses a virtual catalog and product presentation 
method and apparatus which allows an operator to overlay an image of a product on a 
selected background. 

U.S. Patent 5,974,400 to Kagami et al. discloses a system for allowing a person 
to "try on" clothing virtually. 

U.S. Patent 6,144,388 to Bornstein discloses a system for displaying articles of 
clothing on an image of a person. 

U.S. Patent 6,281 ,903 to Martin et al. discloses a method and apparatus for 
embedding 2D image content into 3D models by encoding a specification of a two 
dimensional background image into a three dimensional foreground object that overlies 
and occludes a portion of the background. 

U.S. Patent 6,522,787 to Kumar et al. teaches rendering a synthetic image from 
a mosaic of a scene based on a viewpoint, and combining the synthetic image with a 
second image to generate a composite image containing a realistic combination of 
objects in the second image and the scene. 
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4700. 
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